These are the results for the performance tests:

Generated by 🚫 dangerJS against 5a7905e

This pull request has conflicts, please resolve those before we can evaluate the pull request.

I have pushed a simplified version of the demo, aiming to be as close as possible to how developers might fetch the data server-side. It fixes the double fetch flickers. However, it seems that we have a couple of issues. A summary of what I think is off, by order of importance (to complement the existing open comments on lines of code):

1. When using the filter in the demo, there are a couple of strange behaviors: flickering of row rendering, crashes of out-of-range row updates, and crashes of duplicated React keys. I don't know why.
2. The onFetchRows API itself has no notion of cancellable requests. It assumes that calls are resolved instantly. But if you fetch, sort, and request a previous sortModel resolves, it breaks your data. So IMHO, this can't be used in production by a developer. The issue can be seen when filtering the demo. The results displayed are the ones from an incomplete search because they resolve after the query will the full request. AG Grid has this notion https://ag-grid.com/javascript-data-grid/server-side-model-datasource/#success-callback. I guess it could be handled by the demo, maybe we should turn to RxJs.
3. onFetchRows is called every time a new row is in the viewport, so when you scroll, you can easily throw 50 requests to your backend. So IMHO, this can't be used in production by a developer, he would DDoS his backend. I think that we should either scope down the feature, for example, call it something like onRowsViewportChange with a non-primary goal of data fetching or we should batch the calls by introducing a notion of chunk size so it can be used for data fetching. But maybe support of this in the demo might be enough, as an illustration. It will be painful for developers but it's still a step forward. AG Grid has this notion: https://ag-grid.com/javascript-data-grid/server-side-model-api-reference/#reference-serverSideRowModel-cacheBlockSize
4. For the skeleton, add a small delay for appearing, like 100ms, to avoid flashes

I get now the amount of complexity there is in something like https://ag-grid.com/javascript-data-grid/server-side-model/, why it's not MIT.

Ok, I've pushed the solution a bit more and cleaned most of the issues.

1. When using the filter in the demo, there are a couple of strange behaviors: flickering of row rendering crashes of out-of-range row updates, and crashes of duplicated React keys. I don't know why.

Should be fixed now

3. onFetchRows is called every time a new row is in the viewport, so when you scroll, you can easily throw 50 requests to your backend.

I left it like this intentionally. We can add a line in the doc that advises the developers to implement either throttle or debounce themselves. Adding this functionality as part of the feature increases the complexity quite a lot.

4. For the skeleton, add a small delay for appearing, like 100ms, to avoid flashes

I guess this will be added once the Core adds that prop?

2. The onFetchRows API itself has no notion of cancellable requests. It assumes that calls are resolved instantly. But if you fetch, sort, and request a previous sortModel resolves, it breaks your data.

Can't this be solved the same way 3. is?

We can merge this version and see what the feedback will be. We can continue to polish it more but it's been quite a long time and I feel we are operating based on assumptions. Not that it is wrong but I would love to start a clean PR after we get some real-world use cases/bugs 😅

Should be fixed now

@DanailH Great, thanks for looking into it! I have found a few more issues on https://deploy-preview-5214--material-ui-x.netlify.app/x/react-data-grid/row-updates/#lazy-loading.

a. When sorting, the rows updates twice:

b. When opening the filtering panel with an active sort, the rows change:

c. When filtering with an active sort, there are rows that shouldn't match

And I think that it's :). It might be an issue with the demo, and not the source.

Can't this be solved the same way 3. is?

I left it like this intentionally. We can add a line in the doc that advises the developers to implement either throttle or debounce themselves. Adding this functionality as part of the feature increases the complexity quite a lot.

It's definitely more complex, but I also think why we should care. It's precisely for the hard problems that developers will come to use our stuff.

My main concern is: Are we shipping a demo that can't be used in production and with APIs that are not unstable_ so that we can't change without waiting for v6?

From what I understand, if a developer uses https://deploy-preview-5214--material-ui-x.netlify.app/x/react-data-grid/row-updates/#lazy-loading as is, he will DDoS his backend and will show rows to his end-users that don't match what they filter for (out-of-sync). So I would expect that he will need to do further work to solve 2. and 3 on his own. Since we know it, I think that we should fix it before we tell them to use the demo. I'm afraid that if we don't, we might harm the trust that developers have in their ability to copy & paste from the demo in the docs. Since the problem is complex, 💯 to only fix it in the demo. It will give us time to find an API that can then accommodate tree data, pivoting, etc.

I guess this will be added once the Core adds that prop?

We could wait for mui/material-ui#34057. However, I think that it would be great to first dogfood this idea inside MUI X, both to find an implementation that works well, and to prove that it brings value, and that it deserves to be in the core. I think that it could be a follow-up PR 👍 . It's kind of detail, a lot less important than the previous points.

Edit: It seems that this other demo https://mui.com/x/react-data-grid/row-updates/#infinite-loading is also not usable in production as is. The filtering and sorting are not working and the demo is not this straightforward to convert into network API calls. I would be in favor of working on this, considering AG Grid demos seem directly usable.

Alright, I added the last changes:

• add the unstable_ prefix for the replaceRows API
• add a debounce to the onFetchRows callback
• add a warning in the docs to highlight that it's best to implement a debounce so you limit the number of requests since sends to their BE.

Regarding the problems with the demo - I guess it's a limitation with what we have in terms of the fake server. It needs additional polishing but that can be done later on in a clean PR. I will merge this PR as it states it is stable in terms of the lazy loading feature and the replace rows API.

Thank you @oliviertassinari for the detailed review and feedback.